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Since 2003, the NASA Ames Research Center has been actively involved in researching 
and advancing the state-of-the-art of planning and scheduling tools for NASA mission 
operations. Our planning toolkit SPIFe (Scheduling and Planning Interface for Exploration) 
has supported a variety of missions and field tests, scheduling activities for Mars rovers as 
well as crew on-board International Space Station and NASA earth analogs. The scheduled 
plan is the integration of all the activities for the day/s. In turn, the agents (rovers, landers, 
spaceships, crew) execute from this schedule while the mission support team members (e.g., 
flight controllers) follow the schedule during execution. Over the last couple of years, our 
team has begun to research and validate methods that will better support users during real- 
time operations and execution of scheduled activities. Our team utilizes human-computer 
interaction principles to research user needs, identify workflow processes, prototype 
software aids, and user test these. This paper discusses three specific prototypes developed 
and user tested to support real-time operations: Score Mobile, Playbook, and Mobile 
Assistant for Task Execution (MATE). 
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I. Introduction 

N ASA Ames Human Computer Interaction Group, for the last ten years, has developed scheduling and planning 
software tools that support the operations of a diverse set of space missions. We call our scheduling and 
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planning software toolkit SPIFe (Scheduling and Planning Interface for Exploration, pronounced “spiffy”). For each 
space mission, a customized version of SPIFe has been deployed for operations 1 . For instance, the Phoenix 
Scheduling Interface (PSI) was developed and tailored by our group for the Mars Phoenix mission. Similarly, other 
Mars missions (Mars Exploration Rovers, Mars Surface Laboratory) are using versions of SPIFe as part of their 
daily surface operations. While the legacy of SPIFe is rooted in robotic missions 2 , different instantiations of our 
software have recently been deployed to conduct daily operations for International Space Station. 

Much of our work has focused on supporting the task of scheduling and planning, which in the space domain is a 
complex, constrained problem. Our end-users, be it scientists, engineers, or flight controllers, complete this task in a 
timely manner, varying from a few hours to a few days. SPIFe, in general, provides a method of defining activities 
in time alongside an interface and visualization for the collection of activities to be accomplished. Our software 
toolkit has a lot of functionality that facilitates this task, like for example, helping users identify and resolve violated 
constraints that exist in a proposed plan. Many SPIFe users have requested additional aids that would help them 
follow-along the scheduled plan in real-time within the software application itself 3 . 

However, through user observations, we have learned that this strategy, while useful in providing real-time aids 
on a desktop application, limits the number of end-users that could actually use and view the scheduled plan. This 
paper discusses the multiple prototypes we have developed, assessed, and field tested in the last couple of years, 
aimed at providing real-time execution software tools to a broader set of users. We briefly provide an overview of 
current aids that exist in the domain of human space operations, followed by our experience with Score Mobile, 
Playbook, and Mobile Assistant for Task Execution (MATE). 

II. What is in a Plan? 

A. Planning and Schedule for Human Space Operations 

Flight controllers plan for all aspects of mission 
operations for the International Space Station (ISS). 

As an orbiting, complex spacecraft, its operations 
require constant planning in order to support the six 
crewmembers that live on-board. The focus of this 
paper is on the plans created for these astronauts, 
plans that are shared with the flight controller 
community, and finally, extensively used on-board 
the ISS. Astronauts have their schedule carefully 
planned based on a large set of operational 
constraints and objectives set forth by the space 
program. Some of the operational constraints (which 
are also highly interrelated) include communication 
availability, accessible power resources, and the 
spacecraft’s position. Furthermore, activities have to 
be coordinated with all the other international 
partners that help manage the space station. During 
their six-month long stay on ISS, crewmembers 
must maintain the space station and complete a 
large, diverse set of research experiments. Despite seeing the sun rise every 90-minutes, astronaut sustain a sense of 
daily routine, which includes coordinated meal times, night shifts, and extensive exercise routines to counter the 
debilitating effects of long-term effects of microgravity on their bodies. 

The essence of ISS plan for astronauts is relatively straight forward: individual crewmembers have a set of 
activities that must be completed each day. Activities are assigned to one or more astronauts (Figure 1). These 
activities can be: 

• routine, e.g., daily conference with ground, 

• periodic, e.g., replacing worn mechanisms in exercise devices, 

• event-driven, e.g., removing stowage from visiting spacecraft, or 

• mission-specific, e.g., scheduled research experiments or maintenance spacewalks. 

Often, an activity is to be completed by more than one crewmember, requiring astronauts to collaborate. Other 
activities require cooperation of ground controllers, necessitating voice and video communications. Some activities 
can only be done by one crewmember at a time (like for example, using a particular workstation). Coordinating and 
synchronization of multiple schedules is intricate, but essential for ISS operations. 
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Figure 1: ISS astronauts collaborating and 

completing an assigned activity 


Most of the activities scheduled for ISS have associated procedures and instructions for crewmembers. 
Astronauts need specific instruction on how to perform the tasks assigned to them. Unlike spacewalks, most intra- 
vehicular activities are not repeatedly practiced and synchronized to the minute. Astronauts are given skill-based 
training to execute most maintenance tasks and background science training for specific research experiments. 
However, for many non-routine activities, astronauts refer and depend on procedures. The procedures are step-by- 
step instructions to complete tasks, often with specific notes (referred to as execution and ops notes) associated with 
that day’s activity. Additionally, many activities have stowage “notes” associated with them, which is a list of all 
items and their location in ISS that are necessary to complete the given task. 

While the plan itself may appear to be simple, it is developed over many weeks of coordination, careful planning 
and scheduling, resulting in a plan that abides to the operational and programmatic constraints. Once a schedule is 
agreed upon all the international partners as well as two NASA centers, any change to the plan must be reviewed 
and approved by the appropriate set of flight controllers. Changes can be as large as reassignments or additional 
activities to as simple as updates to procedures or execution notes. 

Once the plan “goes live,” i.e., the day of execution, everyone from crew to flight controllers view the schedule. 
Viewing the schedule is a critical part of daily ISS operations. Crew access the timeline and associated procedures in 
order to execute all their assigned tasks in a timely manner, respecting the scheduled constraints. The ground follows 
along the schedule, not only to keep situational awareness of what is occurring in the space station, but also if 
needed for coordinated activities. Crew provides feedback on the timeline as to their progress and completion of 
tasks, which in turn flight controllers access and use as a means of collecting data about the scheduled activity. 

1. Visual Layout of a Plan 

Currently, ISS mission operations is updating their scheduling and planning software tools. Score (a version of 
SPIFe for ISS crew planning, Figure 2) supports the planning phase, during which flight controllers schedule 
activities for six astronauts. Once a plan is completed, it is used and seen by crew and flight controllers via a web 
viewer interface. The viewer is a separate software tool (i.e., not part of Score); other web aids are used to connect 
activities to their associated procedure. 

Figure 2 depicts the typical visual layout of a schedule. This layout is unusual when compared to common 
calendar arrangements, though perhaps not so different to program management scheduling. Each crewmember has 
a “band”, a horizontal layout of each scheduled activity. All crewmembers’ schedule are stacked, allowing for 
simultaneous viewing of everyone’s activities at any one point in time. Similarly, there are horizontal “bands” for 
important conditions, such as when ISS is scheduled to be in daylight or when voice communication is available. 
The use of color is driven by various operational reasons, including depicting groupings of activities or the need to 
complete that activity at precisely the designated time. 



Figure 2: Score user interface, with a day ? s schedule for six astronauts 
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B. Earth-Analogs for Human Space Exploration 

The NASA Ames HCI Group has contributed to advancing the state of the art of scheduling and planning, and 
we have started researching how to improve upon supporting real-time operations and execution of planned 
schedules. In order to investigate and collect data on the best methods to support real-time execution, we have 
strategically conducted user testing and field tested software prototypes of timeline viewers to Earth-analog sites. 

Earth-analogs are simulated exploration missions organized by NASA (or NASA-sponsored) aimed at 
investigating different aspects of deep-space human exploration. Score has been used as the planning and scheduling 
tool for a variety of these earth- analog missions. We have also had the opportunity to conduct research at the 
following analogs: Pavilion Lake Research Program (PLRP), Desert Research and Technologies Studies (RATS), 
and NASA’s Extreme Environment Mission Operations (NEEMO). 

Each of these analogs emphasizes a different aspect of exploration to simulate, and typically, their 
“crewmembers” are mostly composed of current astronauts. In PLRP, the focus is scientific exploration in an 
extreme environment. Scientists use a combination of divers, piloted/personal submarines, and remote control 
underwater robots to explore lakes in British Columbia in search of freshwater microbialites. Desert RATS 
emphasizes field testing new technologies and mission operations strategies, in the context of planetary habitats and 
a variety of transportation architectures, from rovers to asteroid-grappling spacecrafts. NEEMO, having an 
underwater habitat, allows for six crewmembers to live in a confined space for two weeks while completing a series 
of research experiments. For all analogs, a daily schedule must be defined and executed, and all team members view 
and share the same timeline. 


III. Score Mobile 


A. Driving User Needs 

Our data collection and user observations started during the 2010 analog season, where we observed crew 
members and support members of analogs having a difficult time following along with the current and upcoming 
activities scheduled for the current day. The integrated activity plan, developed by the mission planners, was only 
available in Score, which was only installed on the workstations located in the crew’s habitat and on workstations in 
the Mobile Mission Command Center (MMCC). Additionally, easy access to these workstations was not feasible. 
Often, crew members were executing activities or moving between locations to perform their activities. While away 
from a workstation, analog crewmembers were not aware of how much time was being spent completing the current 
activity or how soon the next activity would start. As a result, crewmembers had to the additional burden of 
returning to a workstation or use communication channels to ask planners in the MMCC about the plan. Both of 
these methods disrupted the completion of tasks, potentially impacting the rest of the activities scheduled for the 
day. 

These observations led to the first driving user need 
that has guided the rest of our research: the need for a 
mobile, light-weight viewer. Score, being a desktop 
application, is not suited nor designed to currently 
support this need, hence, Score Mobile was developed. 
The primary goal of Score Mobile was to give crew and 
the analog support team members access to upcoming 
activities scheduled for the day through their existing 
smart phones and mobile devices (such as tablets). A 
walk-up-and-use interface was designed to fit the needs 
of analog team members, allowing them to quickly 
peruse the plan and understand what activities were 
currently occurring as long they had internet access. 


B. Prototype Capabilities 

The Score Mobile development team utilized the 
emerging mobile web framework, JQuery Mobile and 
the Score client to quickly develop a web application. 
Figure 3: Score Mobile interface design Score Mobile was used in three space analogs in 2011, 

PLRP, Desert RATS, and NEEMO. For each subsequent 
analog, Score Mobile was adapted for the specific needs and improved based on observation and user feedback. 
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The initial release displayed activities from the plan in chronological order in a vertical list (Figure 3). 
Throughout the day, the list updated to show the current activities at the top highlighted in yellow. Users could pick 
from several timeline bands to customize the display. Each timeline band contained the activities for a single crew 
member or an analog support team. An overview timeline band was also created to allow team members to 
understand the general themes of the day. Upcoming activities showed a count-down timer, indicating the amount of 
time until the next activity. Users could find further information about activities’ details as well as access a PDF 
copy of the entire schedule. Score Mobile also displayed notifications to the whole team which served as a simple 
group text message system, including notifications for when a new, updated version of the day’s plan had been 
uploaded. 

Score Mobile was first deployed in the Pavilion Lake Research Project (PLRP) analog. Team members used the 
mobile website to follow the activity plan, check on submarine flight paths (embedded map images), and share 
announcements with team members. We learned many important lessons from this analog and collected feedback 
from the support teams. One of the critical lesson was that crewmembers wanted more than just the basic details of 
an activity; crewmembers also wanted access to the execution procedures, which were not linked through Score 
Mobile. This additional capability was provided to Score Mobile in support of the Desert RATS campaign (Figure 
4). 



Figure 4: Desert RATS crewmembers using Score Mobile in the rover and inside habitat 


Post-analog questionnaires were completed by crewmembers, providing us with Score Mobile feedback. An 
unexpected finding was that Score Mobile was also used by crew members to review upcoming plans during off- 
hours. Crew members appreciated the ease of use of Score Mobile and the fast access to the PDF timeline available 
in their mobile devices. One important feedback indicated that improvements could be made to better illustrate 
durations of activities. In Score Mobile, the vertical layout of a scheduled depicted a 5 minute activities in the same 
way as 5 hour activity. Users could not quickly assess the duration or 
the time left in an activity by just glancing at the timeline. 

Additionally, users wanted more control over which activities were 
focused on. When an activity took longer than expected, the activity 
would scroll off the page, crew members were unable to change the 
plan or change the status of activities to indicate they were taking 
longer than expected. 

Another key feedback provided by the RATS support operations 
team and crewmembers was the value of seeing the “as run” plan in 
real-time. This particular feature was implemented for the NEEMO 
analog, making it the primary interface for both the support team and 
the crewmembers. Score Mobile was integrated into Score to allow 
support teams to view the as run plan alongside upcoming changes to 
the plan,. Planners were given a “post to Score Mobile” shortcut 
button in Score, permitting quick updates of the plan to the website 
throughout the day, such that team members would always get the 
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Figure 5: Score Mobile in NEEMO's 
MMCC 



latest version of the plan. Score Mobile was displayed and used by crewmembers who were in the underwater 
habitat and by the support team in MMCC (Figure 5). We received similar feedback with respect to ease of use, but 
an important, additional feedback provided by the users of Score Mobile at NEEMO was that crew wanted to also 
easily communicate plan updates to the surface team. 


IV. Playbook 

One essential data collected during the 2011 analog season was the need to support real-time crew feedback 
through Score Mobile. Most of our efforts had focused on pushing the planned schedule out to various, diverse set of 
users, but we did not consider nor facilitate new scheduling information being pushed back to the mission planners. 
Unfortunately, the developed infrastructure of Score Mobile did not easily allow for that capability. A new mobile 
web application was needed, and hence, Playbook was developed. Designed and developed from the data collected 
from previous analog seasons, incorporating lessons learned from Score Mobile, Playbook would serve as a central 
location for crew members to access plan, and procedure data, as well as allow the crew to communicate with the 
rest of the analog team. 

A. Driving User Needs 

Like Score Mobile, Playbook was built as a situational awareness tool for both crew and support teams. Building 
upon the positive feedback and the needs identified by analog crewmembers as well as support teams, Playbook 
design and implementation focused on the following five key needs: 

1. Tablet-focused interactions. In 
analogs, crew had shown a strong 
preference for using iPads to view 
schedule and procedures. Interviews 
with ISS crew had shown similar 
requests. Score Mobile had been 
designed primarily for use on 
phones, which meant limiting the 
amount of information on the 
screen. A design aimed at tablets 
needed to be much richer, showing 
items like multiple individuals’ 
schedules on the screen. While we 
wanted to address the platform 
preference, we also knew that 
Playbook must scale to a variety of 
devices, as the support teams 
preferred using laptops and phones. 

2. A visual timeline. Score Mobile 
feedback indicated that durations of 
current and future activities was still an important piece of visual information that this prototype lacked. 
Users asked for a timeline view that showed all activities happening in parallel, akin to the visual 
presentation in Score (Figure 2) and current ISS schedule viewers. A shift to tablet- focus platforms 
provided us with the ability to include this feature. 

3. Ability to edit the schedule. In dealing with complex schedules, analog crew quickly discovered the plans 
would change on-the-fly. To increase situational awareness, and to make sure everyone was on the same 
page, crew requested the ability to edit the schedule — indicating status changes to activities (in progress, 
complete, aborted, etc) or rescheduling the activities to new times. 

4. Live Updating. One of the key findings from Score Mobile was the importance of keeping everyone 
involved in the mission on the same page. Playbook needed to allow for the above capabilities while 
constantly updating to any changes made by support teams. When this was not possible, such as in adverse 
network conditions, it was important to notify people that their information was out of date. 

5. Embedded procedures. Originally, Score Mobile was not designed as an execution aid. Improvements were 
made to Score Mobile to address this but Playbook was designed with the idea of increasing crew 
efficiency by imbedding materials required for execution within the schedule itself. To increase crew 



Figure 6: Playbook showing an analog scheduled timeline 


6 

American Institute of Aeronautics and Astronautics 


efficiency, procedures and supporting documents such as stowage notes need to be quickly accessible, 
ideally embedded within the schedule itself. 

B. Prototype Capabilities 

Playbook shows a full timeline view, showing bands for each user and any supporting role (Figure 6). The 
schedule is imported from Score, easily refreshed, and reflects the schedule that was carefully constructed by the ops 
planner. This tightly-knit connection is essential, allowing the ops planners to have control of the schedule and data 
the rest of the analog team views. By default, the view shows a single day at a time. The ’’day picker” shortcut (in 
the top right) can be used to switch between days. The zoom buttons can alter the amount of information shown and 
on touch-screen devices, an intuitive pinch and zoom gesture can be used as well. 

Tapping an activity opens an area on the side called ”the stream” (Figure 6). This is a focused view of a single 
band. Depending on how a band was defined, it can show all the activities associated with a person, place, or piece 
of equipment. Selecting one of these activities in the stream puts it into focus. Each activity can then be updated by 
the user, for instance, updating the activity’s status or add notes to the activity. Equally, tapping individual activities 
in the timeline allows users to update them. This two-way selection is important — allowing users to quickly switch 
between seeing the details of an activity and seeing how that activity fits into the bigger context of the mission. 

Once an activity is selected, any associated procedures appear inline. Selecting the name of the procedures opens 
a new browser tab with a detailed step by step walkthrough of the procedure. This "new tab" behavior allows 
multiple procedures to be opened at the same time, permitting crew to reference multiple documents, if necessary, in 
complex situations. 

In addition to the schedule, other information is available through the master navigation toolbar (Figure 7). The 
"Procedures" section is a focused list of all the procedures across the activities. The "Mission Log" evolved over the 
course of several analogs. In early iterations, the mission log was a running log of all the changes happening in the 
mission. Users in the NEEMO analog found this useful, but overwhelming: the log would quickly fill with crew 
marking activities as "complete". Notes and aborted activities would get buried. The concept was revamped to be an 
off-band communication channel with rich media (Figure 8). Users could send a picture, video, or text note and it 
would automatically appear on everyone's screen without interrupting the on-going voice loops. In the RATS analog 
this proved to be especially useful in sending science updates, e.g. new traverse maps for the crew. 




= Playbook 

01 - MMSEV HARDWARE 

I 1.101 MMSEV Standard 
I Operating Procedures, Volume 1 


1.102 MMSEV Trash Operations 


1.103 MMSEV WCS Operations 

1.104 MMSEV Air Conditioning 
Adjustment 

1.105 MMSEV C02 Alarm 
Response 

1.106 MMSEV PWD Dispense 


vt pp (o 




Figure 7: Playbook as viewed in on mobile device. Navigation toolbar shown on left, procedure 
accessibility shown on right. 


C. Future Work 

Playbook was used in 4 space analogs in 2012, with continuing refinement between each. We received positive 
feedback from analog crewmembers, who described Playbook as highly useful to their day-to-day operations. Some 
comments included: “ Mission enhancing tool — essential ” and “Everything you need in a quick to touch format .” As 
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in our previous experience in analogs, crewmembers and analog support teams suggested improvements. Crew self- 
scheduling is still a desired functionality. Early prototypes of Playbook have begun to explore this by introducing 
the ability to highlight activities that can be moved without resource impacts, allow users to drag them around, and 
send the new schedule back to ground. Future work will look at allowing larger degrees of flexibility and doing live 
resource calculations to show the exact impact of changes. 

Our experience in earth-analogs and feedback provided by crewmembers indicate that rich, interactive 
procedures is a foreseen, natural progression to current established procedures. Current procedure technology is 
reaching its limits — crew must follow long-written descriptions of procedures steps and have an especially difficult 
time jumping between procedures. To deal with the former, Playbook has tested embedding videos and audio 
directly into procedures, potentially simplifying procedures for crewmembers. In the future, Playbook may introduce 
and support the concept of M compose-able” procedures, where multiple procedures are embedded together on a 
single page to form a master procedure, eliminating the need to jump between disparate files. Some additional 
improvements in this area include automatic timestamp logging and integrated telemetry. 

In the same vein, multimedia in communication loops seems to be desired future functionality, particularly in 
mission architectures where communication is conducted in an asynchronous manner, like it was in 2012 RATS 
Earth-analog. The acceptance and positive feedback received on the ’’mission log” used at the RATS analog 
validates the need for an asynchronous communication channel that includes a variety of inputs, whether it be video, 
audio, or text. This notion of a “mission log” is not foreign to mission operations, as flight controllers monitor 
communication voice loops. Future mission logs could be akin to multimedia communication loops. Including this 
capability in Playbook will better serve analogs that are simulating future, long duration and long distance 
exploration missions. 

Sampling summary for Site 3, from DSH 

Download: 2.104 sample data sheet 8 27 12 site 3.xlsx 

EVA (VR Lab EV1) 

Are we there yet??? 

MSG-026 Science Sampling Update! 

Download: msa-026 test dav 8 site c update.Ddf 

MSG-025 Science Sampling Update! 

Download: msa-025 test dav 8 site 3 uodate.odf 

Figure 8: Playbook ? s Mission Log 


V. MATE 


To explore mobile-focused software to support operations execution, the HCI Group mentored a student research 
and prototyping project with Carnegie Mellon University. The result was a capstone project and the development of 
the Mobile Assistant for Task Execution, or MATE, which focused on supporting complex task execution through 
procedures within the context of asynchronous communications for space operations. The MATE research is 
different from Score Mobile and Playbook as we were able to research analogous domains, derive lessons learned 
from these, and investigate their application to the space domain through user testing. We observed and/or 
interviewed automotive technicians, surgeons, construction managers, and theatrical stage managers, in addition to 
NASA-associated end-users, including former astronauts and crew members of the NEEMO program. Based on this 
research, we focused on a few key findings from which three primary design objectives emerged for MATE: provide 
intuitive support aids for learn-ability, allow users the flexibility to handle interruptions, and present all data 
associated with a procedure in one, minimally-cluttered screen. 
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A. Driving User Needs 




B. Prototype Capabilities 

MATE prototype has four key features: the crewmember’s list of daily activities (the ’’home view”), a dedicated 
activity view for each activity (the ’’activity view”), persistent note-taking area, and a ground communication panel. 
The home view gives an overview of the crewmember’s day as well as relevant contextual information, including 
recent notes, scheduled lapses in communication, and the most recent daily planning summary. The activity view 
provides the user with all of the information needed to execute a scheduled activity. Features such as progress- 
marking and step folding help reduce cognitive load during execution, while a built-in timer helps the crewmember 
stay on schedule without feeling rushed. The note-taking interface lets users create notes for long-term 
communication as well as make them available for later use. The notes could be at associated with the whole day or 
comment can be connected to just a specific activity step. Finally, ground communication was implemented through 
a text-based message center, permitting crewmembers and ground crew to communicate with one another 
asynchronously. Non-intrusive visual notifications informed users of incoming messages. 


Haw 

18 M DPC 


MONDAY DAY 14 


Ultrasound 2 Hardware Setup and Power On 


Daily Planning Conference 

objective: This is the objective. 


UPCOMING COMM LOSS 


UPCOMING TIME CRITICAL TASK 


D COMM I 


START ACTIVITY 


Routine Changeout of filters for In-Ground Return. 


Small filters should be used for the return air filter 
on the Overhead Vents. Large filters are used for 
the In-Ground Return. You will be changing the 
filter in an In-Ground Return. 


TIME CRITICAL AT 07:25 


Morning Prep Work 

objective: Preparation Work 


Integrated Cardiovascular Echo Ultrasound 2 Pre-Scan 

objective: Crew sets up COL video camera and config- 
ures Ultrasound 2 for ICV Echo scan. Use of COL port , 
camera for cabin video is assumed unless crew directs... 


Integrated Cardiovascular Echo Ultrasound 2 Pre-Scan 

OBJECTIVE: Crew sets up COL video camera and config- 
ures Ultrasound 2 for ICV Echo scan. Use of COL port } 
camera for cabin video is assumed unless crew directs... 


Great job on the video link with ground yesterday! Those fifth graders seemed 
really inspired! Also, the full waste water compartment is causing high humidity 
in the EMCS incubator which is undesirable from an engineering point of view as 
it may cause corrosion and/or malfunction to the different components inside 
the incubator over time. 

For Lance: As we review the ATV logs from yesterday, we see what may be a 
discrepancy on date/time on some entries. Would you please check for accurate 
data and time and report on the results? 


Assembling rod structures is tricky. 
Too much off nominal use of tape 
required in this activity. 

Friday, Day 11 RSAM-ASSMBLE : 


Remember to use scissors to cut the 
tape. Tearing leads to uneven strips 
of tape and make the structure 
harder to assemble. Assembling rod 
structures is tricky. Too much off 
nominal use of tape required. 


Thursday, Day 10 


M-ASSMBLE 


Assembling rod structures is tricky. 
Too much off nominal use of tape 
required in this activity. 


Remember to use scissors to cut the 
tape. Tearing leads to uneven strips 
of tape and make the structure 
harder to assemble. Assembling rod 
structures is tricky. Too much off 
nominal use of tape required. 

Wednesday, Day 9 RSAM-ASSMBLE > 


Assembling rod structures is tricky. 
Too much off nominal use of tape 
required in this activity. 


Tool Name 

Camera 

Rod Structure Assembly M. 
Red Rod 
Green Rod 
Yellow Rod 


Quantity Location 

1 Bin D9 

1 Module W7 

1 Sack B6 

1 Sack B6 

1 Sack B6 


CHECK TO SEE IF CAMERA H 


ASSEMBLE THE STRUCTURE 


Serial Number 


HTP448 

HTP448 

HTP448 

HTP448 

HTP448 


2.i Verify that rod receptacles (refer to Figure 1) are complete. 


» 


0 



Figure 9: MATE, home view (left) and activity details view (right) 


C. User Testing Results 

MATE went through five prototypes iterations, each with increasing fidelity (i.e., from paper to interactive PDF 
to iPad application). Usability testing at each round provided foci for the next iteration. Early tests were aimed at 
creating a visual language for displaying the steps involved in task execution. These initial tests led to features 
specifically aimed at maintaining visual clarity, such as progress tracking and a communication log that hides itself 
when not needed. 

By having users conduct activities of increasing complexity throughout development, MATE was able to 
communicate the complex, information-rich instructions (ISS-like procedures) without cognitive overload. We were 
able to demonstrate this by recruiting naive participants and asking them to use MATE after a short ten-minute 
training session. These users were asked to complete a nonlinear series of objectives designed to exercise 
implemented features, giving the user the freedom to transition between tasks and interface tools as they saw fit 
(Figure 10). These scenarios tested MATE’S core features, each designed to be as intuitive as possible to reduce 
cognitive load in high-stress, interruption-prone environments. Test participants exhibited near-perfect performance 
(with respect to completing the assigned tasks), despite multiple interruptions and shifts in procedure context. This 
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Figure 10: Participant executing complex task 
using MATE 


combination of familiarity and simplicity also allowed 
users to use MATE in unexpected ways, developing 
unique workflows and strategies in situ. 

Due to its limited development cycle, MATE 
ultimately did not include every feature that would be 
desirable for task execution onboard the ISS. Note-taking 
with rich media, for instance, was high on every user’s 
list of desired features but was deemed too technically 
difficult for this prototype. Issues also arose with user 
compliance, specifically in the area of progress tracking. 
Despite these areas not fully explored, MATE was a 
fundamental success in its ability to enable the essential 
communication and task execution needed to conduct 
complex procedures in space. 


VI. Concluding Remarks 

Field-testing software prototypes in Earth-analogs has permitted exploration of new ways of providing support 
for real-time space mission operations and execution of scheduled activities that currently are not in place for ISS 
operations. Our research experience shows that mobile devices have become an essential component to sharing and 
viewing schedules among a large team. This conclusion is not too surprising considering how ubiquitous mobile 
devices are now-a-days. Furthermore, multiple types of mobile platforms (e.g., phones, tablets, laptops) have to be 
supported as various end-users have different needs. Our research also indicates that it is essential to design schedule 
viewers for multiple mobile platforms from the beginning, as retrospectively incorporating this capability is 
technically challenging and results in missing functionality. 

One design challenge we constantly had to address for with our real-time execution prototypes was the need to 
implement a software aid that was easy to “pick up and use.” For Score Mobile, Playbook, and MATE, there was 
minimal to no training for these tools, yet they had to serve the needs of a very diverse set of end users. One set of 
users wanted to a quick, easy method of obtain the necessary situation awareness of current and future activities. 
Aside from simply viewing the timeline, most of these users have expectations of receiving summary information, 
as in a “feed”. All of our prototypes supported this, be it through an overview “band”, the mission logs, or the 
asynchronous text updates from “ground”. 

Another set of end users, namely, crewmembers, aside from wanting a mobile viewer and needing situation 
awareness of the overall day’s schedule, also require much more detailed information about the day’s activities. 
Much of our research focused on learning about their needs, as these users are different from the typical mission 
operations planners that operate SPIFe. User tests of Score Mobile, Playbook, and MATE has shown that 
crewmembers need software aids to support the following three aspects of real-time execution: how to complete a 
task, how long do they have to complete it relative to the rest of the schedule, and how to communicate to others 
progress. Moreover, these aspects should be tightly knitted together in their design and implementation in order to 
be effective. In every prototype described, we tried to better integrate procedures into activities, evolving with each 
iteration. One aspect which we can improve upon still is providing crewmembers better indications of progress and 
remaining duration, ideally combining these with progress or notes on procedures and in turn, automatically 
communicating this to ground controllers. 

Finally, with regards to supporting real-time communication, be it between crew to ground, or amongst flight 
controllers, the need to support a diverse set of communication protocols was prevalent. Texting or chatting is just 
one simple addition to the traditional voice communication. However, now-a-days, using multimedia, i.e., video, 
images, is common, and often requested in the operations of Earth-analog missions. Our prototype tools were quick 
to adapt and will continue to evolve in order to meet this necessity. 

One aspect that we are still working towards in the future is to better support scheduling communications 
between crew and ground. At a minimum, astronauts want to provide completion updates to flight controllers. Yet, 
Earth-analog crew request and future long-duration missions will need to support rescheduling requests. Any future 
real-time operations and execution software aid will have to allow for end-users to request or communicate changes 
in the planned schedule. One simple example is the use case where a required activity takes longer to complete than 
scheduled. Astronauts will want to communicate progress and see the impact to the rest of their schedule, requesting 
changes based on the in situ information they have. Such an aid needs to be tightly integrated with the planning and 
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scheduling tool in order to assess if any constraints are not being fulfilled. This work is both a technical and design 
challenge and easily communicating this information back and forth is not straight-forward. 

Over the course of a couple of years, the HCI group has learned much about the needs to support real-time 
operations and execution of scheduled plans, through mobile devices that allow for situation awareness and activity 
execution while incorporating flexibility to facilitate various end-users with different types of objectives. Working 
alongside the operations mission planners in the Earth-analogs has permitted this iterative process of developing and 
field testing software prototypes with crewmembers and ground controllers. This research allows us to explore the 
next aspect of scheduling and planning, enabling future software aids for long-duration human space operations. 
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